Although this invention is being disclosed in connection with cervical cancer, it is applicable to many other areas of medicine. Uterine cervical cancer is the second most common cancer in women worldwide, with nearly 500,000 new cases and over 270,000 deaths annually (IARC, “Globocan 2002 database,” International agency for research in cancer, 2002, incorporated herein by reference). Because invasive disease is preceded by pre-malignant Cervical Intraepithelial Neoplasia (CIN), if detected early and treated adequately, cervical cancer can be universally prevented (D. G. Ferris, J. T. Cox, D. M. O'Connor, V. C. Wright, and J. Foerster, Modern Colposcopy. Textbook and Atlas, pp. 1-699, American Society for Colposcopy and Cervical Pathology, 2004, incorporated herein by reference).
An automated image analysis system of uterine cervical images analyzes and extracts diagnostic features in cervical images and can assist the physician with a suggested clinical diagnosis. Such a system could be integrated with a medical screening device to allow screening for cervical cancer by non-medical personnel. The system also has potential applications in the screening of, for example, female soldiers, marines and sailors who are deployed in locations where annual Pap testing is not possible. Further, such a system has tremendous potential benefits for screening underserved women in developing countries.
The purpose of a cervical screening method is to identify and rank the severity of lesions, so that biopsies representing the highest-grade abnormality can be taken, if necessary. The examination involves a systematic visual evaluation of the lower genital tract (cervix, vulva and vagina), with special emphasis on the subjective appearance of metaplastic epithelium comprising what is known as the Transformation Zone (TZ) on the cervix. During the exam, a 3-5% acetic acid solution is applied to the cervix, causing abnormal and metaplastic epithelia to turn white (“acetowhite”). Cervical cancer precursor lesions and invasive cancer exhibit certain distinctly abnormal morphologic features that can be identified by the visual examination. Lesion characteristics such as color or opacity, margin shape, blood vessel caliber, intercapillary spacing and distribution, and contour are used by physicians to derive a clinical diagnosis (R. Reid, C. P. Crum, B. R. Herschman, Y. S. Fu, L. Braun, K. V. Shah, S. J. Agronow, and C. R. Stanhope, “Genital warts and cervical cancer. III. Subclinical papillomaviral infection and cervical neoplasia are linked by a spectrum of continuous morphologic and biologic change”, Cancer, 53, pp. 943-953, 1984, incorporated herein by reference). Lugol's iodine is another contrast solution often used during the exam. The color difference of the iodine staining also assists in differentiating the severity of the lesions.
Similarly, the color and illumination of the cervical images vary with the light sources, the instruments and camera settings, as well as the clinical environment. Consequently, the color of the cervical epithelium may look very different (including normal and abnormal findings) in cervical images acquired with different instruments or at different times. This makes the assessment of the color information very challenging, even for an expert. Using an objective image calibration technique (accompanied by corresponding monitor calibration technique) may help the physician to better assess the information in cervical images in terms of diagnosis and severity, for improvement of the annotation and the use of telemedicine.
The use of digital imaging is revolutionizing medical imaging and enabling sophisticated computer programs to assist the physicians with Computer-Aided-Diagnosis/Detection (CAD). Clinicians and academia have suggested and shown proof of concept to use automated image analysis of cervical imagery for cervical cancer screening and diagnosis (B. L. Craine and E. R. Craine, “Digital imaging colposcopy: basic concepts and applications”, Obstetrics and Gynecology, 82, pp. 869-873, 1993, incorporated herein by reference; W. Li, V. Van Raad, J. Gu, U. Hansson, J. Hakansson, H. Lange, and D. Ferris, “Computer-aided Diagnosis (CAD) for cervical cancer screening and diagnosis: a new system design in medical image processing”, Lecture Notes in Computer Science, CVBIA 2005, pp. 240-250, 2005, incorporated herein by reference; M. S. Mikhail, I. R. Merkatz, and S. L. Romney, “Clinical usefulness of computerized colposcopy: image analysis and conservative management of mild dysplasia”, Obstetrics and Gynecology, 80, pp. 5-8, 1992, incorporated herein by reference). Various image processing algorithms have been developed to detect different colposcopic features, such as acetowhite color (S. Gordon, G. Zimmerman, and H. Greenspan, “Image Segmentation of Uterine Cervix Images for Indexing in PACs”, in Proceedings of IEEE 17th Symposium on Computer-based Medical Systems, 2004, incorporated herein by reference; H. Lange, “Automatic detection of multi-level acetowhite regions in RGB color images of the uterine cervix”, in Proc.SPIE, 5747, pp. 1004-1017, SPIE, San Diego, 2005, incorporated herein by reference; and S. Gordon, G. Zimmerman, R. Long, S. Antani, J. Jeronimo, and H. Greenspan, “Content analysis of uterine cervix images: initial steps towards content based indexing and retrieval of cervigrams”, in Proc.SPIE, 6144, pp. 1549-1556, 2006, incorporated herein by reference), lesion margin (I. Claude, R. Winzenrieth, P. Pouletaut, and J.-C. Boulanger, “Contour Features for Colposcopic Images Classification by Artificial Neural Networks”, in Proc of International Conference on Pattern Recognition, pp. 771-774, 2002, incorporated herein by reference; V. Van Raad, Z. Xue, and H. Lange, “Lesion margin analysis for automated classification of cervical cancer lesions”, in Proc.SPIE, 6144, 2006. incorporated herein by reference), and blood vessels (Q. Ji, J. Engel, and E. Craine, “Texture Analysis for Classification of Cervix Lesions”, IEEE Transactions on Medical Imaging, 19, pp. 1144-1149, 2000, incorporated herein by reference; Y. Srinivasan, D. Hernes, B. Tulpule, S. Yang, J. Guo, S. Mitra, S. Yagneswaran, B. Nutter, B. Phillips, R. Long, and D. Ferris, “A probabilistic approach to segmentation and classification of neoplasia in uterine cervix images using color and geometric features”, in Proc.SPIE, J. M. Fitzpatrick and J. M. Reinhardt, Eds., 5747, pp. 995-1003, 2005, incorporated herein by reference; and W. Li and A. Poirson, “Detection and characterization of abnormal vascular patterns in automated cervical image analysis”, Lecture Notes in Computer Science: Advances in Visual Computing, 4292, pp. 627-636, November 2006, incorporated herein by reference). On the other hand, lack of color calibration makes it very difficult to extract the color property of the acetowhite lesions properly. Non-uniform illumination and light distribution also has been a major obstacle in extracting lesion margins and blood vessel structures compared to the colposcopic annotations.
CAD on cervical imagery could have a direct impact on improving women's health care and reducing the associated costs. Accurate color calibration is a crucial factor in developing a CAD system for cervical imagery. Several image enhancement techniques, such as histogram stretching and/or equalization, have been used as an attempt to compensate for the illumination problem (Y. Srinivasan, D. Hernes, B. Tulpule, S. Yang, J. Guo, S. Mitra, S. Yagneswaran, B. Nutter, B. Phillips, R. Long, and D. Ferris, “A probabilistic approach to segmentation and classification of neoplasia in uterine cervix images using color and geometric features”, in Proc.SPIE, J. M. Fitzpatrick and J. M. Reinhardt, Eds., 5747, pp. 995-1003, 2005, incorporated herein by reference; S. Yang, J. Guo, P. King, Y. Sriraja, S. Mitra, B. Nutter, D. Ferris, M. Schiffman, J. Jeronimo, and R. Long, “A Multi-Spectral Digital Cervigram™ analyzer in the wavelet domain for early detection of cervical cancer”, in Proc.SPIE, J. M. Fitzpatrick and M. Sonka, Eds., 5370, pp. 1833-1844, 2004, incorporated herein by reference).
Generally speaking, the colors in an image depend on the light source, the image acquisition device, and the properties of the subject being imaged The red, green, and blue (RGB) color filters of a digital color camera are designed to mimic the color sensitivity of the human eye and are, thus, said to be creating a “true” color image. In reality, the color filter responses are fairly dissimilar to the sensitivity of the human eye, which means that color cameras and the eye represent colors quite differently. Different color representations are especially noticeable under different lighting conditions. Consequently, depending on lighting conditions and camera characteristics, digital color images often are different from what is perceived by human eye. Because colors are very important to how we perceive the world around us, people have studied these differences, including ways to correct them, in great detail. Making the same image look identical, independent of the camera, monitor or printer used, has been a desired outcome ever since the advent of photography. The goal of the present invention and image calibration in general, is to make the colors of a cervical image appear identical, independent of camera settings and light source. This is preferably achieved by mapping the color appearance of the images taken with different instruments into a standard color space, as illustrated in FIG. 1.
Although many of the algorithms described in the present invention are well-known in the art, the inventors are unaware of another simple and robust, color calibration system that both corrects for non-uniform illumination and calibrates the color of images using only one uncalibrated light source, one uncalibrated standard visible light detector, and one color target. The present invention uses the ground truth and native reflectivity (described below) of only the targets, and maps the native reflectivity back to the ground truth reflectivity (described below) without any knowledge of the light source, detector or the environment. The following patents and patent applications may be considered relevant to the field of the invention:
U.S. Pat. No. 7,012,633 to Jenkins, incorporated herein by reference, discloses a color calibration method for an imaging color measurement device utilizing a detector array, a plurality of optical elements, and multiple instances of irradiation of the detector array for a single measurement. A flat-fielding correction error correction matrix of the imaging color measurement device for each instance of irradiation of the detector array is obtained prior to color calibration. The response for each instance of irradiation of the detector array is flat-fielded with the corresponding error matrix to obtain a flat-fielded, spectrally weighted irradiance response for each instance of irradiation of the detector array. An illuminant light source with known spectral output or chromaticity coordinates is measured to obtain an irradiance response of the imaging color measurement device for each instance of irradiation of the detector array. A color correction coefficient is calculated using the known spectral output and chromaticity coordinates of the light source and the corresponding flat-fielded irradiance response.
U.S. Patent Publication No. 2007/0142707 to Wiklof, et al., incorporated herein by reference, discloses an endoscope system and method for providing images of anatomical features imaged using the endoscope system. The system also includes a calibration device having known optical properties. The calibration device is imaged using the endoscope system, and data corresponding to the image is obtained. This data are compared to data corresponding to the known optical properties of the calibration device. Based on this comparison, calibration data corresponding to imaging errors of the endoscope system are obtained and the calibration data are used to calibrate the endoscope system.
U.S. Pat. No. 6,147,705 to Krauter, et al, incorporated herein by reference, discloses a video colposcope which includes a system microcomputer having algorithms for color balance levels stored into memory. A video camera obtains a subject electronic image of a subject object, and using algorithm-driven digital signal processing circuitry (DSP), color saturation, hue, and intensity levels of the subject electronic image are modified according to DSP reference filter algorithm and reference color balance levels as stored, thus producing a modified electronic image corresponding to the subject electronic image. The modified electronic image is outputted to a display in continuous real time as the corresponding subject image is obtained by the video camera. This modified electronic image emulates that obtained through an optical green filter and incorporates a simulated white balance.
U.S. Pat. No. 5,016,173 to Kenet et al., incorporated herein by reference, discloses an improved apparatus and method for in vivo monitoring of visually accessible surfaces of the body. The invention synthesizes methods of systems identification and computer vision to quantify and/or classify features of surface or subsurface anatomic, physiologic, or pathologic structures or processes. Such is accomplished by the stimulation of anatomic surfaces with light (visible, infrared, and/or ultraviolet, structured or uniform), followed by the quantitative analysis of digital images (multiresolution, multiview, and/or multispectral) of reflected or emitted light from the surface of interest.
U.S. Pat. No. 5,836,872 to Kenet, et al., incorporated herein by reference, discloses a method for monitoring a region of a body surface including a method for diagnosis of a premelanomatous or early melanomatous conditions. The color calibration technique used, images a color chart, or set of light emitting diodes, of standard known colors, either during a calibration session or during the acquisition of images of the surface feature under examination. Regions of the image containing known colors may be used to identify the set of pixel values representing that color. This set of pixel values (e.g. reds green and blue pixel values) for an individual known color may then be used to determine input look-up table values, or pixel scaling factors to apply to all pixels of an image that will result in standardization of color between images obtained under similar lighting conditions.
U.S. Pat. No. 6,101,408 to Craine et al., incorporated herein by reference, discloses an apparatus and a method for determining the area of a three-dimensional lesion on a cervix from a two-dimensional image of the cervix.
U.S. Pat. No. 5,791,346 to Craine et al., incorporated herein by reference, discloses an apparatus and method for accurately computing an area on a three-dimensional object from a two-dimensional image data obtained by means of a camera, such as one associated with a digital colposcope.
U.S. Patent Publication No. 2006/0241347 to Whitehead, incorporated herein by reference, discloses an systems and methods relating to colposcopic viewing tubes for enhanced viewing and examination.